Mixing Rates for Gibbs Sampling
ثبت نشده
چکیده
Proof. Let x, y 2 ⌦ be two configurations. We will prove the claim for the visible conditional distributions. The proof for the hidden conditional distributions will follow symmetrically. For each visible node v i , let (X(v i ), Y (v i )) be the maximal coupling of P (v)(X(v i ) |x(h)) and P (v)(Y (v i ) | y(h)) guaranteed in Lemma 1. By doing this independently for all visible nodes, we have a valid coupling (X,Y ) of P (· |x(h)) and P (· | y(h)). Then we can work out the expected Hamming distance of X and Y as E[d v (X,Y )] = n X
منابع مشابه
Mixing Rates for the Alternating Gibbs Sampler over Restricted Boltzmann Machines and Friends
Alternating Gibbs sampling is a modification of classical Gibbs sampling where several variables are simultaneously sampled from their joint conditional distribution. In this work, we investigate the mixing rate of alternating Gibbs sampling with a particular emphasis on Restricted Boltzmann Machines (RBMs) and variants.
متن کاملMixing Rates for the Gibbs Sampler over Restricted Boltzmann Machines
The mixing rate of a Markov chain (Xt)t=0 is the minimum number of steps before the distribution of Xt is close to its stationary distribution with respect to total variation distance. In this work, we give upper and lower bounds for the mixing rate of the Gibbs sampler over Restricted Boltzmann Machines.
متن کاملEnsuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling
Gibbs sampling is a Markov chain Monte Carlo technique commonly used for estimating marginal distributions. To speed up Gibbs sampling, there has recently been interest in parallelizing it by executing asynchronously. While empirical results suggest that many models can be efficiently sampled asynchronously, traditional Markov chain analysis does not apply to the asynchronous case, and thus asy...
متن کاملRapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width
Gibbs sampling on factor graphs is a widely used inference technique, which often produces good empirical results. Theoretical guarantees for its performance are weak: even for tree structured graphs, the mixing time of Gibbs may be exponential in the number of variables. To help understand the behavior of Gibbs sampling, we introduce a new (hyper)graph property, called hierarchy width. We show...
متن کاملA slowly mixing Markov chain with implications for Gibbs sampling
We give a Markov chain that converges to its stationary distribution very slowly. It has the form of a Gibbs sampler running on a posterior distribution of a parameter f3 given data X. Consequences for Gibbs sampling are discussed.
متن کاملRapid Mixing Swendsen-Wang Sampler for Stochastic Partitioned Attractive Models
The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the SwendsenWang dynamics which is a more sophisticated Markov chain designed to overcome bottlenecks that impede the Gibbs sampler. We pr...
متن کامل